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Abstract 

This paper contains research results from five individual projects to characterize the 
spatial performance of the IKONOS commercial imaging sensor. The end result of the 
projects is determination of the spatial image quality of IKONOS data products in terms 
of the National Imagery Interpretability Rating Scale (NIIRS), the system Modulation 
Transfer Function (MTF), the system stability over the first year, the characteristics of the 
Space Imaging MTF Compensation (MTFC) procedure, and the application-specific 
capabilities of IKONOS imagery. Both panchromatic and multispectral imagery were 
evaluated. Major conclusions of this work are that the system was stable in imaging 
performance during the first year of operation- that its MTF meets the specification for 
the NASA Scientific Data Purchase program, that the initial MTFC processing appears to 
be transposed in the in-track and the cross-track directions, that the MTFC results in a 


noise amplification of 2x to 4x in addition to sharpening the imagery, and that IKONOS 
panchromatic imagery achieves an average NIIRS rating of 4.5. 

1. Introduction 

The pre-eminent characteristic of IKONOS is its significantly higher spatial 
resolution compared to other non-military satellite remote sensing systems. Both the 
National Aeronautics and Space Administration (NASA) and the National Imagery and 
Mapping Agency (NIMA) have been purchasing IKONOS 1 m and 4 m ground sample 
distance (GSD) imagery for various purposes. NIMA acquires commercial satellite 
imagery as part of its mission to provide geospatial information to the Department of 
Defense (DoD) and to the national intelligence community. NIMA’s vehicle for assessing 
the image quality and utility of commercial imagery is the Civil and Commercial 
Applications Project (CCAP). NASA Stennis Space Center purchases imagery primarily 
for land use research, and its academic partners at the University of Arizona and South 
Dakota State University are responsible for assessing image quality for NASA as part of 
the Scientific Data Purchase (SDP) Verification and Validation. 

Although NASA and NIMA address significantly different sets of problems, both 
agencies are using this high-spatial-resOlution imagery in similar ways. While the 
multispectral aspects of satellite systems have historically been exploited by NASA 
researchers, in the case of IKONOS, the 1 m panchromatic and pan-sharpened 
multispectral imagery have been used primarily for visual inspection and mapping 
applications, similar to the way such imagery is used by the defense communities (Garvin 


et al., 2002; Tucker, 2002). NASA scientists rely upon the IKONOS spatial and 
geolocation characteristics primarily to detect and to identify small features . 

The work presented here illustrates some of the complexities encountered when 
dealing with an emerging commercial product. While not commonly known, NASA 
negotiated its contract and data specifications with Space Imaging, LLC before the 
system was on-orbit and before all the present product options were available. NIMA 
negotiated its contracts with Space Imaging later resulting in different products being 
available to NASA and NIMA. One of the significant differences in offerings is that 
NIMA has the ability to purchase imagery that is radiometrically corrected without 
geometric correction, while NASA purchases only imagery that has been both 
radiometrically and geometrically corrected. The NIMA contract with Space Imaging 
calls the pure radiometric products “TIFF” and the resampled products “GeoTiff.” At the 
time of the NASA contract negotiations, only cubic convolution resampling was 
available. NIMA contracted with Space Imaging to produce a nearest neighbor resampled 
product that later was made available to the NASA community. 

Users of IKONOS imagery should investigate the relative benefits of each post 
processing option. The radiometry paper in this issue (Pagnutti et al., 2003) discusses the 
effects of Modulation Transfer Function Compensation (MTFC) on radiometry in various 
scene types. In this section, we focus on the spatial domain effects of the MTFC option. 
NASA was also initially offered imagery that had MTFC applied. MTFC is a form of 
image sharpening that attempts to correct the inherent Modulation Transfer Function 
(MTF) roll-off with spatial frequency caused by finite detector size, spacecraft motion, 
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diffraction, aberrations, atmospheric scattering, turbulence, and electronic effects (Holst, 
1995 ). 

One of the more interesting imaging product options has MTFC applied. Often, 
MTFC is used to boost the National Imagery Interpretability Rating Scale (NIIRS) rating 
and image interpretability. Applications that depend upon spectral analysis generally 
require higher Signal-to-Noise Ratio (SNR) than do purely visual applications. For both 
types of analyses, it is important to understand the effect of MTFC on SNR, as well as the 
spatial frequency content of the scene being studied, before selecting the MTFC 
processing option. In the sections below, we estimate the magnitude of effects produced 
by MTFC processing. 

Another issue not generally known is that all the data are compressed, using a Kodak 
proprietary compression technique, off the focal plane from 1 1 bits/pixel to 2.6 bits/pixel 
for transmission to the ground. In the strict sense, this nonlinear compression violates the 
linear shift invariant requirement for MTF analysis. For these reasons, the results 
described in this paper should be interpreted as product-specific and are not the true 
fundamental engineering performance of the system. Product evaluations are becoming 
more common as we use additional commercial imagery sources and produce more 
complex products from government systems. 

The intent of the spatial characterization effort funded by NAS A-Stennis Space 
Center, under the Science Data Purchase program, and by NIMA is to: 

• Evaluate the usefulness of IKONOS for image interpretation tasks 

• Understand the types of spatial processing available from Space Imaging 
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• Evaluate the on-orbit spatial imaging performance of IKONOS 

• Determine if any degradation has occurred during the first year of IKONOS 
operation 

Research results that address these goals are presented in this paper. A variety of 
approaches are used including visual inspection, modeling, noise analysis, image 
gradients, and MTF. In Section I, we describe the relationships between image quality, 
edge response, and imagery options such as MTFC to the National Imagery 
Interpretability Rating Scale. NIIRS is a graduated, criteria-based, ten point scale used to 
indicate the amount of information that can be extracted by imagery (IRARS, 1996). In 
Section II, the University of Arizona describes relative image quality analysis using the 
image Digital Number (DN) gradient as a sharpness measure. In Section III, South 
Dakota State University (SDSU) describes MTF analysis using rectangular “pulse” 
targets. In Section IV, NASA describes MTF analysis using an “edge” target at the 
Stennis Space Center (SSC). In Section V, NIMA describes the application of IKONOS 
imagery in standard image interpretation tasks and NIIRS estimation with certified 
analysts. 

Finally, we summarize the entire paper and tie together the various analyses and 
attempt to draw useful conclusions for future sensor characterizations. 
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2. Image quality in terms of NIIRS and edge response assessment estimates 

The spatial resolution of most remote sensing systems is described in terms of the 
sensor MTF and GSD. In the case of IKONOS, the imagery NASA purchased was 
specified to have a GSD at nadir of 0.82 m in the panchromatic band and of 3.24 m in the 
multispectral bands. In addition, the minimum allowable MTF at the Nyquist frequency 
was specified to be 0.1 in the panchromatic band and 0.24 in the multispectral bands. 
These specifications, however, are for raw data and not for the products available to both 
NASA and NIMA. 

Image quality is the result of a complex relationship between GSD, MTF, MTFC, and 
SNR. MTFC generally increases the sharpness and interpretability of the imagery, but it 
also introduces several artifacts, such as ringing at edges and increased noise. Visual 
interpretability ratings such as NIIRS can be estimated from the edge response, ringing 
overshoot, and SNR using the empirically based General Imagery Quality Equation 
(GIQE) (Leachtenauer et al., 1997). An edge response is determined from an image of 
extended bright and dark uniform rectangular areas of at least 10 x 20 pixels in extent or 
is estimated from a pulse or other target. The edge response is normalized such that the 
asymptotic dark and bright values are scaled to zero and unity, respectively. An important 
image quality metric used in the GIQE is the relative edge response (RER). The RER is 
the slope of a normalized edge, measured at +0.5 pixels from the edge center location. A 
commonly accepted form of the GIQE that accounts for the effects listed above follows: 

0 344G 

NIIRS = l0.25l-alog l0 GSD GM +blog l0 RER GM -0.656H GM -^^- (1) 
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Where GSDgm is the geometric mean of the ground sampled distance, RERgm is the 
geometric mean of the relative edge response, H G m is the geometric mean-height 
overshoot caused by MTFC (Leachtenauer et al., 1997), and G is the noise gain 
associated with MTFC. In the current form of the GIQE, SNR is estimated for differential 
radiance levels from Lambertian scenes with reflectances of 7 percent and 15 percent 
with the noise estimated from photon, detector, and uniformity noise terms. If the RER 
exceeds 0.9, then a equals 3.32 and b equals 1.559; otherwise, a equals 3.16 and b equals 
2.817. 


The GSD is computed in both ground plane directions in inches, from which GSDqm 
is then calculated. Similarly, the RERgm is the geometric mean of the RERs computed in 
the orthogonal image directions. The GIQE overshoot H accounts for the ringing 
associated with the MTFC and is measured over 1 .0 to 3.0 pixels from the edge in 
0.25-pixel increments. In most cases, the overshoot H is the maximum value over this 
range. However, if the response is monotonic over this range, the overshoot H is taken as 
the value at 1.25 pixels from the edge. Again, H is estimated in orthogonal image 
directions and the geometric mean is calculated. The noise gain term is defined in 
Equation 2: 


N N 


G = 


MTFGkerml-j ) 2 
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Noise gain associated with the MTFC processing was investigated using simulated 
scenes with the expected noise properties of IKONOS imagery and estimating the root 
sum square of the MTFC kernel provided by Space Imaging. Table 1 lists the estimated 
noise gain associated with each band. The panchromatic noise gain is approximately 4, 
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being twice that of the multispectral bands with the multispectral band noise increasing 
with wavelength. The simulation results and expression (2) are in good agreement. 
Although Space Imaging implements its MTFC option in the spatial domain, the MTFC 
is described here in the frequency domain as provided by Space Imaging to NASA for 
analysis. MTFC functions are designed to boost the higher spatial frequencies without 
impacting the zero spatial frequency. The IKONOS panchromatic MTFC function is a 
monotonic function increasing from unity at zero spatial frequency to over 6 at the 
Nyquist frequency. This function is calculated by zero-padding a fast Fourier transform 
of the MTFC kernel. The function is a relatively symmetrical flower petal shape, but it is 
slightly stronger in the cross-track direction than in the in-track direction. Although not 
shown here, NASA also derived the MTFC by estimating the transfer function from 
several georeferenced scenes, with and without MTFC. The magnitude and shape of the 
MTFC functions qualitatively agreed with the functions provided by Space Imaging, but 
NASA found that the transfer functions were always rotated from true north by several 
degrees or more because Space Imaging used a kernel that combines the resampling and 
MTFC processing. 

2d MTFC simulated e dg e effects 

Simulated edge responses were generated assuming a symmetrical Gaussian MTF 
with an MTF at the Nyquist frequency of 0.1, the IKONOS MTF specification for 
panchromatic imagery. A Gaussian MTF function was chosen because many system 
MTF functions are approximately Gaussian in shape. NASA did not consider any 
resampling in these simulations. The expected asymmetries were ignored because NASA 
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lacked sufficient system information to model them accurately. The simulations are thus 
approximations, but they do illustrate several features seen in other sections of this paper. 
The simulations start with an 8x oversampled edge response that is then decimated to the 
supplied sampling. In the case of panchromatic imagery, a simulated edge is generated at 
a 0.108 m GSD for the simulations and then resampled to 0.82 meters to allow 
examination of the effects of sampling and aliasing on the edge response. A second set of 
edge responses was generated by applying the MTFC processing. In the real imaging 
process, the individual detectors sample a continuous function with the relative sampling 
controlled by the focal length, slant range, and detector spacing, while the absolute 
position relative to an edge is a random process. This level of oversampling minimizes 
any aliasing effects in the simulation. The magnitude of the ringing depends on the 
phasing of the sampling. The overshoot was about 1 0 percent and the RER improved by 
approximately 50 percent. 

The multispectral MTFC functions are not as strong as the panchromatic MTFC; their 
peak values are less than one half the panchromatic MTFC peak. This is not surprising 
since the IKONOS multispectral specification for MTF at the Nyquist frequency is 0.24. 
Also, the MTFC functions are stronger as wavelength increases. The overshoot is under a 
few percent for all bands. Each MTFC function should be different for each multispectral 
band, since both diffraction and charge coupled device MTF functions are wavelength 
dependent. The multispectral MTFs are also far more asymmetric than the panchromatic 
MTFC function. The near- in frared (MR) MTFC function is the most symmetric and the 
blue band is the most asymmetric, with each MTFC function being stronger in the cross- 
track direction than in the in-track direction as seen in the blue band edge response 
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simulations. This calculation is similar to panchromatic calculations but with the 
multispectral MTF at the Nyquist frequency set to 0.24 for the simulations. The MTFC 
processed edge showed only minimal change in the in-track direction. Because the MTF 
typically rolls off more quickly in the in-track direction, the MTFC was expected to be 
stronger in the in-track direction. At the time of this writing, NASA and Space Imaging 
had discussed this finding, and Space Imaging is considering rotating the kernels. 

3. Multispectral and panchromatic band relative image analysis 

3. 1 Introduction 

Constructed and cultural targets can be used for MTF analysis of satellite imaging 
sensors (Schowengerdt et al., 1985; Rauchmiller and Schowengerdt, 1988; Storey, 2001). 
Examples of target use for MTF analysis of IKONOS are included in the SDSU and SSC 
sections of this paper. However, logistical difficulties with constructed targets and lack of 
control of cultural targets, such as bridges, roads, and other linear features, make target- 
free approaches to image-quality evaluation desirable. This section describes such an 
approach undertaken from July 2000 to July 2001 to determine IKONOS image quality 
stability. 

3.2 Rationale 

Two images of the same area taken at different times will have unique 
characterizations due to changes in surface cover, solar illumination angles, sensor view 
angles, atmospheric conditions, and sensor performance. If the non-sensor-related factors 
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are negligible for a given pair of images, then it would be possible to use the image pair 
for an evaluation of change in sensor performance. The DN gradient (i.e., the discrete 
derivative between neighboring pixels) is proposed as an image quality metric. It is well 
known that the DN gradient is directly related to the “sharpness” of an image. The image 
pair to be compared is first processed to extract the same ground region and to match the 
DN histograms on a global basis (Schowengerdt, 1997). The latter step removes global 
(consistent over the whole image) solar irradiance variations due to different solar angles 
and global atmospheric transmittance or path radiance differences on the two dates. 

The DN gradient calculation, we chose the Roberts gradient, the magnitude of the 
vector sum of neighboring pixel differences in the ±45° directions, which is a long 
established way to obtain an image gradient (Castleman, 1996). 

3.3 Sensitivity 

Two examples will be used as reference benchmarks for this image comparison 
technique. Both are panchromatic images processed without MTFC and with MTFC. One 
image is of the Big Spring, Texas, MTF target maintained by Space Imaging for 
IKONOS evaluation, and the other image is of Tucson, Arizona. The differences are 
visually substantial in both cases as shown in Figure 1 . The target image represents a 
relatively simple scene consisting almost entirely of edges and lines. The Tucson image is 
a more complex scene consisting of cultural features similar to those in the anniversary 
image with which it was compared (the only difference was some offset due to different 
frame locations). Because the image collects are the same in both cases, there are no 
differences other than those caused by MTFC. Comparison of the average gradient 
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yielded the results in Table 2. Therefore, average DN gradient differences on the order of 
30 to 45 percent are expected in comparison of MTFC-on and MTFC-off processed 
images. 

3.4 Image quality stability 

Two IKONOS collects were obtained over Tucson, Arizona, on July 23, 2000, and on 
July 15, 2001. The solar and sensor angles for these collects are given in Table 3. Solar 
azimuth and elevation affect the length and direction of shadows and affect the overall 
irradiance of level terrain by the cosine of the solar zenith angle (Schowengerdt, 1997). 
The effect of sensor azimuth and elevation on the recorded image is more complex. For 
example, in the case of high-resolution sensors such as IKONOS, sensor azimuth and 
elevation can determine whether one side of building is seen. In addition, specular 
reflection can occur from small, directional objects, such as vehicle windshields and 
metal roofs. In this case (Table 3), the cosine irradiance factor between the collects in 
2000 and 2001 is 1 .0348, or only about 3 percent. This factor was applied as a gain to the 
2000 image to normalize it to the same average irradiance as the 2001 image. 

Two areas were extracted from each year’s image and the Roberts gradient was 
applied to each area. The areas were chosen to avoid significant land cover changes (e.g., 
new pavement on a parking lot or road) and were registered visually when cut from the 
images. One of the characteristics of this technique for image comparison is that it is not 
particularly sensitive to image misregistration of a few pixels. Since a large area is used, 
the average gradient magnitude is little affected by a slight offset in either image. The 
average DN gradient magnitudes calculated from the image regions are given in Table 4. 
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The average percentage difference on the two dates was 4.6 percent for Area 1 and 8.9 
percent for Area 2. In both cases, these differences are much less than the nearly 50 
percent difference between the MTFC-off and MTFC-on images for the Big Spring target 
and Tucson data (Table 2). 

3.4 Summary and conclusions 

A simple relative-analysis technique for measuring sharpness was applied to two 
IKONOS collects of Tucson, Arizona, taken approximately a year apart. Using the 
average gradient magnitude as a measure of image sharpness, the two images differ by 
less than 10 percent. Relative to the gradient magnitude difference of 30 to 45 percent 
between images with and without MTFC, the two images acquired approximately a year 
apart have the same image quality. Other than imaging system performance changes, 
factors that could cause the 10 percent difference include differences in sensor look 
angles, changes in land cover, and differences in solar irradiance angles. These factors 
were minimized in this study by using images with high sensor elevation angle, by 
selecting image regions with little land cover change, and by using image collects with 
solar elevation and azimuth angles within 5° of each other. 

4. On-orbit MTF measurement by SDSTJ 

4. 1 Introduction 

This section describes a procedure that was developed for estimating the Edge Spread 
Function (ESF) and MTF of high spatial resolution imaging sensors while in orbit. No 
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underlying mathematical model is assumed since complete system descriptions for 
typical sensors are rarely available. 

4.2 Method description 

The primary target consisted of a set of 3 m x 30 m blue tarps placed in a relatively 
uniform grassy field and oriented in a 2 x 4 pattern representing a rectangular shape 
12 m x 60 m in size. The 60 m length extended from north to south as shown in Figure 2. 
As shown in the figure, these tarps are quite bright in the blue with approximately 0.4 for 
reflectance. As wavelength increases, the reflectivity decreases until reflectances in the 
red wavelengths are nearly the same as vegetation. However, in the NIR, reflectance 
again increases to nearly 0.4. Thus, this target is particularly well suited for MTF 
evaluation of multispectral imagery in the blue and in the near infrared. 

Tarp 1 and tarp 2 (T1 and T2) were selected as reference tarps. They were aligned by 
surveyor’s transit at an angle of 8° east of true north to obtain as straight an edge line as 
possible. In addition, all seams were aligned by transit to maintain straight edges. To 
understand better the importance of target angle, an example ESF is shown in Figure 3. 
All pixel centers are shown as dotted angled grids. The dashed lines indicate the phasing 
of the pixel center locations as the edge location changes with each row of pixels. The 
horizontal axis is scaled in units of pixels corresponding to one ground sample interval in 
the output image. The vertical axis, in units of digital number, represents the value of 
each pixel. The output edge function is then sampled at a resolution of 20 points per 
pixel. As the orientation of the angle changes, the resolution changes as well, becoming 
either coarser or finer. Optimal angles exist that place the subpixel sample locations on a 
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uniform grid with the limited target length. Because IKONOS imagery is resampled so 
that true north is “up,” an optimal target angle for both panchromatic and multispectral 
bands was found to be 8° east of true north. 

This method’s first step involves determining exact edge location. Edge positions are 
determined on a line-by-line basis using available pixel information. For example, a 
blurred edge is shown in Figure 4. Simple digital differentiation has been applied to 
detect maximum slope. The subpixel edge points are determined by fitting a cubic 
polynomial to the edge data using four values around the maximum slope point. The zero 
crossing location of the second derivative indicates the curve’s inflection point, which is 
then assumed to be the subpixel edge location. 

An underlying assumption is that the edge of the target lies in a straight line. Any 
deviations from a straight edge represent errors in the geometry of the image and a 
potential contribution to the overall MTF of the system. With this thought in mind, all 
edge cross-sections were forced to lie along a straight line by fitting a least squares line 
through the subpixel edge locations obtained from the previous step and then declaring 
that the actual edge locations lie on that line. In Figure 5, the circles on the edge show the 
input edge positions for individual rows of pixels. The line represents the least squares 
estimate for all edge positions. Cubic splines were then used to interpolate each usable 
horizontal row of “aligned” edge data. Twenty values were interpolated within one pixel 
point to build a pseudo-continuous line. All vertical rows were used to estimate one 
averaged spline as shown in Figure 6. 

The tarp width should be carefully chosen because of the zero crossing points in the 
sine function. This is especially important since many imaging systems are required to 
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meet minimum specifications of MTF at the Nyquist frequency. A tarp width of one pixel 
does not include any zero crossing points before the Nyquist frequency; however, the 
output signal from such an input is too small and may likely be affected by noise. With 
tarp widths of two, four, or six pixels, the Nyquist frequency occurs at the zero crossing 
point; when the output FT is divided by the input FT, the MTF value at the Nyquist 
frequency cannot be determined. To compute MTF values at the Nyquist frequency, a 
tarp width of three pixels appears to be optimal. Although a three-pixel wide input 
contains one zero crossing point between zero and the Nyquist frequency, the width is 
large enough to produce a well-defined image target and the output signal is strong 
enough to be minimally affected by noise. 

4.3 Results 

As shown in Figure 7, the width of the pulse was determined by the actual tarp width. 
Edge detection was applied on every row, and the subpixel edge positions were adjusted 
by the least squares fitting line. The average profile was normalized by the difference 
between the mean grass DN value and the mean tarp DN value. The input pulse and the 
adjusted output were then Fourier transformed (Figure 8); the curve denoted with stars is 
the discrete Fourier transform of the output and the plot with circles is the Fourier 
frahsfonh of the input pulse. Finally, the MTF was calculated by dividing each output 
value by its corresponding input and by normalizing the result by the spatial average 
component magnitude. The resulting MTF, shown in Figure 9, has a value at the Nyquist 
frequency of approximately 0.31. 
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One of the drawbacks of working with pulse type targets is that in Fourier space, the 
corresponding function is a sine function that has a number of zero crossings. In a 
noiseless system, the output response would also be zero at these frequencies. However, 
because noise is always present in real systems, significant errors will often be introduced 
at frequencies near where these zero crossings occur. Because the input function is 
rapidly approaching zero near the zero crossing frequency, division in Fourier space will 
often produce noticeable errors in MTF at frequencies close to where the zero crossings 
occur. An apparent anomaly observable in Figure 1 1 at the normalized frequency 0.3 
where the MTF function value is larger than what the overall MTF curve suggests it 
should be. 

Figures 10 and 1 1 exhibit all four pulse spread functions and MTFs obtained during 
the summer 2001 season. Figure 10 shows a consistent estimate of the pulse spread 
function with undershoot visible on each side of the pulse. Figure 1 1 shows good 
repeatability of MTF estimation with the single exception noted above. Estimates near 
the input sine function zero crossing frequency (i.e., between f=0.3 and f=0.4) show more 
variability. 

4. 4 Summary 

This work was an attempt to characterize the performance of a high-spatial-resolution 
imaging system in orbit by estimating its frequency response to ground inputs. The tarp- 
based target provided a usable pulse input for the four-meter multispectral bands. The 
physical layout of the target was found to be critical for a reasonable MTF estimation; 
proper orientation of the target enabled reasonable determination of subpixel edge 
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locations. The MTF results obtained from IKONOS images suggested that the min i m um 
value at the Nyquist frequency for the multispectral bands was 0.27, exceeding NASA’s 
Scientific Data Purchase specifications. 

5. MTF analysis using the Stennis Space Center target 

5. 1 Introduction 

Spatial resolution of image products is affected not only by characteristics of the 
satellite camera but also by processing of the images after reception at a ground station. 
Image processing may include such steps as geometric correction and geographic 
registration as well as image sharpening based on MTF compensation. Because the 
spatial resolution characterization is conducted for the on-orbit satellite, the imaging 
process is also affected by transmission of radiation through the Earth’s atmosphere. 
Therefore, characterization of the spatial resolution of image products involves 
evaluation of the entire remote sensing system. 

Full width at half maximum of a line-spread function is used as a measure of spatial 
resolution of the images. Before LSFs are derived from edge responses by numerical 
differentiation, the edge responses are measured and analyzed using a modified knife- 
edge technique (Tzannes and Mooney, 1995). Adjacent black and white square panels, 
either painted on a flat surface or deployed as tarps, form a ground-based edge target used 
in the tests. During the measurements, the edge target is intentionally oriented so the 
image of the edge is aligned slightly off-perpendicular to a pixel grid direction. The 
tilted-edge modification to the original knife-edge method allows properly sampled edges 
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to be obtained, minimizing aliasing (Reichenbach et al., 1991). To measure an edge 
response, a rectangular region containing the tilted edge is extracted from an image of the 
edge target as shown in Figure 12. In such a region, each line across the edge forms an 
approximate edge response. Exact edge responses (in the direction perpendicular to the 
edge) are obtained when distances are additionally scaled by cosine of the tilt angle. The 
distance correction is usually small, but it becomes important when results from 
measurements with different edge orientations are to be compared. 

Size of the edge target panels is a critical factor in spatial resolution measurements of 
satellite images. To accommodate the 1 -meter GSD of the IKONOS panchromatic 
images, panels 20 m by 20 m in size are used. Larger panels would provide even more 
accurate measurements, but deployment of such large targets becomes extremely 
difficult. Therefore, the number of edge response samples available for analysis is still 
limited by the target size even with the additional sampling provided by the edge tilt. For 
panels of given size, sampling of the spatial response is also affected by the edge tilt 
angle. When deployable tarps are used in the measurement, orientation of the edge can be 
optimized for maximum oversampling. Based on pre-test simulations, a tilt angle of 5° 
was selected and used in measurements conducted with a set of specially coated, 
reflective tarps owned and operated by the NASA Earth Science Applications Directorate 
at S tennis Space Center. 

With a limited number of the available edge response samples, image noise may 
significantly affect the results of a spatial resolution characterization. Not only does 
numerical differentiation of an edge response amplify the noise present in the data and 
produce a spurious LSF, but also determination of the edge response slope becomes less 
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accurate. To mitigate adverse effects of image noise and limited sampling, a smooth, 
analytical function is fitted to the edge responses. In the present approach, a superposition 
of three sigmoidal functions is utilized. Only three functions are used because too many 
components could cause the analytical function to fit the data points too well and to 
reproduce the noise and errors rather than the actual information. Thus, the fitting is 
performed for all the edge responses simultaneously using the formula in Equation 3 : 


e i (x) = d + Y j - 


a,. 


k = 1 


1 + exp 


x - b x Ai - b 2 


( 3 ) 


The distance x is measured in the direction perpendicular to the edge. The nonlinear 
least-squares optimization is conducted for nine parameters: aj, <22, 03 , cj, C2, C3, bj, b2, 
and d. Expanding on the work of Tzannes and Mooney, position and orientation of the 
edge are found simultaneously with the parameters characterizing spatial resolution: in 
one computational process of a nonlinear least-square fit of the two-dimensional 
analytical function to the intensities in the edge image. The parameters bj , £2, and c* 
are common for all the edge responses, while the difference in the edge position is 
introduced by the edge response index (/) multiplied by image GSD (A). Because all of 
the edge positions are located on a straight line, they are specified with the simple 
formula bj Ai + b 2. Tangent of the tilt angle is equal to the absolute value of the parameter 
bj. To further suppress noise artifacts, all three sigmoidal functions are restricted to the 
same positions of the edge specified by the parameters bj and 62. This assumption also 
ensures that the analytical edge response function is symmetrical. 
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Examples of the measured edge responses and the analytical functions fitted to them 
are shown in Figure 13. The presented analyses were conducted for IKONOS 
panchromatic image products that were georeferenced using the cubic convolution 
resampling and the Universal Transverse Mercator projection with the WGS-84 datum. 
The satellite acquired two source images on different dates: one for measurement of the 
edge response in the easting direction (along the rows of image pixels) and the other for 
measurement of the edge response in the northing direction (along the columns of the 
image pixels). Each of the source images was used to create two different image products 
by processing it either with MTFC-on or MTFC-off. In this way, effects of MTFC on 
spatial resolution of the images can be directly evaluated by studying this set of four 
images. Finding the parameters bj and 62 during the curve-fitting process is equivalent to 
shifting the edge responses to a single reference location so that all the edge points are 
aligned. Superimposing all the shifted edge responses creates a new one with a finer 
spatial sampling as shown in Figure 14. Comparison of the measured edge responses with 
the fitted, analytical functions indicates that the edge responses are asymmetric. The 
asymmetry is noticeable even in the images processed without the MTF compensation. 
For the images with MTFC, the measured edge responses additionally contain apparent 
overshoots and undershoots; The analytical function does not fit those secondary features 
exactly, but it does reflect the general shape of the measured edge response. Nevertheless, 
it is evident that the MTFC greatly improves spatial resolution of the IKONOS images 
and makes the edge responses much steeper. 

After an analytical edge response function is obtained from the best fit, it is 
differentiated numerically to derive LSF and its FWHM as shown in Figure 15. The edge 
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responses were extracted five times from each of the images by independent selection of 
the analysis area. Mean results of the FWHM measurements obtained by averaging the 
five samples are listed in Table 5. To characterize spatial resolution of the IKONOS 
panchromatic images, the FWHMs should be compared with the 1 m GSD of the 
resampled image products. Such comparison shows once again that the MTFC clearly 
increases sharpness of the IKONOS images. In absolute terms, both images acquired in 
the early months of 2002 have rather narrow LSFs even without the MTFC applied. 
These results confirm high spatial resolution of the IKONOS panchromatic images. 

6. Assessing imagery utility for NIMA: IKONOS panchromatic image 
interpretability study 

6. 1 Introduction 

The NIMA Civil and Commercial Applications Project performed an interpretability 
study of IKONOS-2 panchromatic imagery to evaluate the information content of 
IKONOS imagery in support of standard image interpretation tasks supporting military 
and civilian applications. The metrics used to gauge utility in this study were the NIIRS 
and task satisfaction of Essential Elements of Information (EEI). Ah EEI represents a 
request for intelligence information. The EEIs were restated in terms of image 
observables and related tasks. For example, an EEI might ask the intelligence analyst to 
determine the number of long-range tactical aircraft. The observable might be large 
camouflaged fighter aircraft. The task is “detect large camouflaged fighter aircraft.” 
Image task EEIs are associated with a Visible NIIRS level; e.g., a NIIRS 3 image would 
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be able to satisfy all level 3 EEIs. EEIs are derived from a variety of sources, such as 
NIIRS criteria and the NIMA Community Needs Forecast. The EEIs chosen for the 
IKONOS panchromatic evaluation addressed image tasks for NIIRS levels 3 through 6. 

6.2 Methods 

Twenty-four level 1 TIFF panchromatic images were acquired from Space Imaging. 
TIFF products are radiometrically corrected only. Sensor arrays are joined and contrast 
balanced; geometric distortions are not eliminated, and the GSD for each pixel increases 
with distance from nadir (mean collected GSD of 0.97 m for level 1 imagery). Fifteen 
archived IKONOS level 2 GeoTIFF panchromatic images were acquired from the DoD’s 
Commercial Satellite Imagery Library. GeoTIFF image products have been 
radiometrically and geometrically corrected and resampled to the Universal Transverse 
Mercator projection, hence each pixel has been resampled to a uniform 1 -meter GSD 
(mean collected GSD of 0.99 m for level 2 imagery). Imagery collection dates ranged 
from November 1999 to July 2000. 

The image matrix used in this evaluation was intended to provide scene coverage of 
tropical, arid, northern temperate, and southern temperate climate regimes within a single 
season. Images were examined for Order of Battle content, and image subsets were 
selected or “chipped” out. A total of 72 image chips were used in the evaluation: 

46 image chips were created from the 24 TIFF images, and 26 image chips were created 
from the 15 GeoTIFF images. A chipping routine was used to generate the image chips 
and a two-power (2x) enlargement. The 2x version was created for each chip using 
nearest-neighbor resampling and applying an identical histogram stretch as the lx 
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version. Each chip pair (lx and 2x) was rotated to the appropriate cardinal direction that 
best aligned image obliquity to the top of the display. 

The evaluation was conducted at the NIMA Imagery Support and Assessment 
Branch’s softcopy evaluation facility. All evaluation participants used the same 
workstation with a precision color monitor, which was calibrated before the start of the 
evaluation. The monitors were set to a minimum luminance response of 0.10 fL and a 
maximum luminance response of 35.0 fL. Evaluation participants were free to roam and 
zoom at lx or 2x magnification within the image. All ratings were made at 2x. No 
interactive enhancement of the imagery was allowed, and image chips were rendered 
with no additional processing. 

Participants for this evaluation consisted of eight NIMA Visible NIIRS-certified 
imagery analysts. Evaluation participants were experienced with assessing OB as well as 
natural and cultural features on panchromatic imagery. The evaluation procedure 
consisted of each participant’s reviewing a sequence of panchromatic (pan) scenes on the 
softcopy workstation and responding with both NIIRS ratings and task satisfaction 
confidence ratings for each scene. Examples of specific questions used are as follows: 

* Determine the NIIRS rating for this image 

• What is your confidence in your ability to identify areas suitable for use as 
light fixed-wing aircraft (e.g., Cessna, Piper Cub, Beechcraft) landing strips? 

NIIRS ratings and EEI responses were given for each panchromatic scene before 
advancing to the next scene. The Visible NIIRS manual was available for reference 
during the evaluation. The task satisfaction scores are given on a 0-100 confidence rating 
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scale, where 0 means the task cannot be performed and 100 means the task is certain to 
be performed. The analysts were instructed to assume that normal collateral imagery and 
information about the target were available. All ratings and responses were entered by 
way of a graphical user interface sliding bar. 

6.3 Results 

Eight imagery analysts completed the evaluation over an 1 1-day period. The final 
dataset consisted of 72 NIIRS ratings and 250 confidence ratings of the EEI tasks for 
72 image chips. Statistical analysis of the data began with the standard assessment of 
reliability and consistency. The inter-rater correlation, rater-group correlation, and 
Cronbach's alpha were computed to exa m ine consistency and reliability among the 
imagery analysts. The rater-group correlations ranged from .68 to .75 and the alpha was 
.89, indicating a high degree of consistency among the raters. Analysis of Variance was 
used to identify outliers in the datasets. One image was removed from the NIIRS dataset 
and two images were removed from the EEI confidence ratings. These cleaned datasets 
were used for all further analyses. The average NIIRS value for TIFF pan images was 
4.65 based on a population of 46 image chips obtained from 24 image products. The 
average NIIRS value for GeoTIFF pan images was 4.41 based on a population of 26 chip 
sets from 15 image products. 

For these levels of NIIRS for the TIFF imagery, the RER is approximately 0.7. 
Approximately a 0.4 NIIRS improvement is expected with MTFC processing based on 
the GIQE. 
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6.4 N1IRS ratings 


The first step in obtaining NIIRS ratings was to calculate descriptive statistics. Then 
an ANOVA was conducted to examine the difference in ratings for format (GeoTIFF vs. 
TIFF) and climate (Arid, Tropical, Temp (N), Temp (S)), including GSD as a covariate. 
For all analyses that include GSD, the collected GSD was used (as opposed to the 
resampled GSD) and was transformed to logi 0 . Climate was not found to be significantly 
different, so the variable was dropped from further analyses. An Analysis of Covariance 
was conducted to determine the impact of format on mean NIIRS. The analysis included 
log 10 GSD as a covariate, as well as an interaction term between log 10 GSD and format. 

The analysis revealed that format was not a significant main effect, but log 10 GSD (p = 
.02) and the interaction term (p = .07) were significant predictors of NIIRS (R 2 = .19). 
These results indicate that format does not directly impact NIIRS ratings. The 0.24 
difference in mean NIIRS between TIFF and GeoTIFF is based on the average 
differences in GSD and the interaction between GSD and format. TIFF imagery has a 
lower average GSD (.93 1 meters) than GeoTIFF (.993 meters), thus accounting for 
higher NIIRS ratings. These results were used to derive simplified regression Image 
Quality Equations, predicting NIIRS from log 10 GSD for both formats: 

GeoTIFF predicted NIIRS = 4.4 1 - 0.77 • log 10 GSD (4) 

TIFF predicted NIIRS = 4.52 - 5.05 • log 10 GSD (5) 

However, because of the limited range of GSD (0.8 to 1.3 meters), it was not possible 
to fit an IQE to predict NIIRS with great accuracy. The predicted NIIRS was the same for 
both formats at a GSD of 1 .06 meters. It is useful to compare equations (1) and (2) to the 
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GIQE derived to predict NIIRS based on electro-optical parameters and system design 
(General Image Quality Equation (GIQE) User’s Guide, 1996). The slope for the TIFF 
equation is much steeper than the GIQE-obtained -3.32; however, a 95 percent 
confidence interval includes -3.32, indicating that GSD is a significant predictor of NIIRS 
ratings for TIFF imagery. On the other hand, the confidence interval for GeoTIFF 
regression slope includes zero, indicating that GSD may not be a significant predictor of 
NIIRS for GeoTIFF imagery. 

If the slope were indeed zero, then the NIIRS value for this imagery does not depend 
on the collected GSD. Since GeoTIFF images have been resampled to have a processed 
GSD of one meter regardless of collected GSD, a slope of zero is not unreasonable. 
However, the zero slope can be true only over a very limited range of GSD. For example, 
an image with a collected GSD of two meters that has been resampled to one meter 
clearly will not be as good as an image with a true GSD of one meter. Thus it is 
hypothesized that over a larger range of GSD, equation (4) would be a broken line with 
two slopes instead of a straight line, with the break occurring at or near one meter. The 
slope for GSD less than one meter would be zero or possibly somewhat negative. The 
slope of GSD greater than one meter would be comparable to EO panchromatic imagery 
characterized by the GIQE; i.e., about -3.31. 

6.5 Essential elements of information 

Means and standard deviations were calculated for the 250 confidence ratings for 
various EEIs. Each EEI had an associated NIIRS level obtained from a previous 
evaluation. ANCOVAs were conducted to compare confidence ratings by climate, NIIRS 
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level, and format, including log 10 GSD as a covariate. NIIRS levels were determined by 
rounding the NIIRS requirement for each task to the lower level (i.e., a 3.2 and a 3.8 
rating would both be NIIRS Level 3). Mean EEI confidence ratings were analyzed by 
climate, format, log 10 GSD, and NIIRS level and their interactions. Format, level, and 
logioGSD were found to be significant (R 2 = .476, p < .04). Unlike the NIIRS ratings, the 
interaction between log 10 GSD and format was not significant. Next, the main effects for 
the two categorical variables were explored. TIFF had an average of 6.3 points higher 
than GeoTIFF, indicating that raters had more confidence in their ratings for TIFF 
images. 

A Tukey post hoc test was conducted to determine which NIIRS levels differed 
significantly from one another. The test indicated that all paired comparisons were 
significantly different (R 2 = .45, p < .01). Because NIIRS level 3 EEIs are the easiest to 
answer, they had the highest mean EEI confidence ratings, while level 6 EEIs had the 
lowest mean confidence ratings. A confidence rating of 75 percent indicates that the task 
can be satisfied with reasonable confidence. The results indicated that GSD and the 
interaction of the GSD* format term was significant in predicted NIIRS ratings. Thus, the 
.24 mean NIIRS difference between TIFF and GeoTIFF was due to the lower average 
GSD for TIFF imagery. The analyses of the EEI confidence ratings showed two main 
effects for format and NIIRS level. TIFF imagery had higher mean EEI ratings than did 
GeoTIFF imagery. Also, level 3 NIIRS had the highest EEI ratings (indicating the most 
confidence) while level 6 NIIRS had the lowest EEI ratings (indicating the least 
confidence). 
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6. 6 Conclusion 


An average NIIRS rating of 4.5 was achieved with IKONOS pan imagery, so this 
imagery should be able to satisfy intelligence tasks for NIIRS levels 3 and 4. Although 
the highest NIIRS and EEI values were obtained with the TIFF products, the level of 
processing was not found to be a significant predictor of NIIRS in the sample size used. 
As might be expected, GSD was found to be significant predictor of NIIRS for both 
formats. The range in average NIIRS values for the entire sample of TIFF and GeoTIFF 
images was 3.61 to 5.28, with NIIRS values increasing as collected GSD decreased. A 
statistical analysis shows that the IQE for two formats had statistically different slopes. It 
is suspected that a broken-line IQE is more appropriate for the GeoTIFF images, but such 
a model could not be distinguished from a single linear equation with the data available. 
Because TIFF images are higher NIIRS than GeoTIFF images when the collected GSD is 
actually less than one meter, the TIFF image format is recommended for intelligence 
EEI-type application if the user has the option. This study indicates that IKONOS 
panchromatic products can satisfy requirements for NIIRS level 4 imagery. Where the 
best possible satisfaction of image tasks is required, the highest collected resolution (i.e., 
at nadir) IKONOS panchromatic imagery should be acquired. For imagery where 
coordinate embedded pixels are not required (as in GeoTIFF) and submeter GSD is 
assured, TIFF products should be obtained. 

For imagery where coordinate embedded pixels are not required (as in GeoTIFF) and 
submeter GSD is assured, TIFF products should be obtained. 


28 


7. Conclusions and summary 


Several different types of Space Imaging IKONOS products were evaluated for their 
spatial resolution. Overall the image quality has been excellent and as advertised, but the 
variety of options was new to many of the scientists using the data. The spatial resolution 
was also relatively stable over the period of the work. The wide variety of products 
available from Space Imaging and the lack of access to raw imagery also makes 
comparison with system-level specifications difficult. First, the product is compressed 
onboard the spacecraft. Although little degradation in the imagery is noticeable, classic 
linear system analysis definitions do not necessarily hold. Second, the imagery is 
processed with several proprietary algorithms. The georeferenced imagery is also 
resampled with two different kernels: cubic convolution and nearest neighbor. MTFC 
processing, when performed, is integrated with the various resampling methods, which is 
a new paradigm for the government that historically has had extensive oversight of the 
construction of remote sensing systems. For these systems, product characterization 
needs to be considered instead of system characterization. 

MTFC processing improves imagery quality but increases noise. The panchromatic 
imagery noise level is increased by over a factor of 4. Although not many truly dark 
targets have been available for examination, imaging over water bodies, forest, and other 
dark targets without MTFC is probably the best option. Native GSD imagery has better 
interpretability than resampled imagery; however, NIMA is presently the only 
government agency that can obtain this product. 

Comparing results between different groups is challenging because all groups did not 
evaluate the same products. A more intuitive approach is to follow the GIQE, in which 
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edge response, GSD, SNR, and processing methods are used to define the spatial 
resolution requirements in terms of the MTF at the Nyquist frequency. Although the 
GIQE has not been fully tested for the various IKONOS product options, developing 
specifications in terms of the GIQE parameters could be beneficial. 

Currently it is very difficult to measure MTF at high spatial frequencies on-orbit 
because the SNR of such measurements degrades with increasing spatial frequency. 
Because these measurements are often noisy and difficult to interpret, other simpler and 
more intuitive metrics, such as the width of the Point Spread Function, the Line Spread 
Function, or the slope of the edge response, may be more practical and useful. These 
three characteristics are all related by Fourier transforms, so measuring any one 
characteristic allows determination of the other two. 
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Figure Legends 


Fig. 1 . Big Spring target (August 5, 2001) with MTFC-off (left) and with MTFC-on (right). The 
Roberts gradient magnitude images are shown directly below. Note the sharper gradient in the 
MTFC-on case. The amplitude of the gradient is greater in the latter case. 


Fig. 2. Illustration of tarp orientation with respect to true north and corresponding example image. 


Fig. 3. Edge spread function projection from angled ground sample interval points. 


Fig. 4. A blurred edge and differentiation of the edge to locate the point of maximum slope. 


Fig. 5. IKuNOS multispectral image of blue tarps deployed at Brookings, South Dakota, on July 
25, 2001. 


Fig. 6. Pulse response function obtained from blue tarps on July 25, 2001. 


Fig. 7. Input pulse function and output pulse response. 


Fig. 8. Input sine function and output response in Fourier space. 


34 


Fig. 9. MTF function for the blue tarp target on July 25, 2001. 


Fig. 10. Overplot of four estimates of blue band pulse response functions from blue tarps. 


Fig. 1 1 . Overplot of four estimates of blue band MTF functions from blue tarps. 


Fig. 12. IKONOS panchromatic images of the edge target tarps deployed at Stennis Space Center, 
Mississippi, for the easting direction measurement on January 15, 2002, (left) and for the northing 
direction measurement on February 17, 2002 (right). Gray rectangular frames overlaid on the tarp 
images show the areas selected for the edge response analyses. 


Fig. 13. Measured edge responses (left column) and the best fits to them with superposition of 
three sigmoidal functions (right column). Data are for the measurements in the northing direction 
using the IKONOS image acquired on February 17, 2002, and processed either with MTFC-on 
(bottom row) or with MTFC-off (top row). Respective image areas are shown on the left side of 
the figure. 


Fig. 14. Superimposed edge responses and the fitted sigmoidal functions for the IKONOS images 
processed either with MTFC-on (bottom row) or MTFC-off (top row). Northing direction is the 
left column and easting direction is the right column. 
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Fig. 15. Line spread functions derived from the fitted edge responses for the IKONOS images 
processed either with MTFC-on (bottom row) or with MTFC-off (top row). Northing direction is 
the left column and easting direction is the right column. 
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Tables 


Table 1 

Noise gain associated with MTFC processing 


Band 

Noise Gain 

Blue 

1.59 

Green 

1.63 

Red 

1.68 

NIR 

1.81 

Pan 

4.16 


Table 2 


Reference image comparisons for sharpness quality metric 


Location 

Processing 

Average DN gradient 

Average percent 
difference 


MTFG-off 

54.2 


Big Spring* Texas 

MTFC-on 

78.2 

44.3 


MTFC-off 

60.97 


Tucson, Arizona 

MTFC-on 

80.38 

31.8 


Table 3 


Anniversary image pair solar and sensor angles 


Date 

Solar 

azimuth (°) 

Solar 

elevation (°) 

Sensor 
azimuth (°) 

Sensor 
elevation (°) 

July 23, 2000 

113.8 

65.4 

136.1 

84.2 

July 15, 2001 

117.7 

70.2 

276.8 

84.1 


Table 4 

Average DN gradient magnitudes for Tucson sites 


Date 

Area 1 (residential) 

Area 2 (road/industrial) 


499x406 pixels 

424x451 pixels 

July 23, 2000 

167.2 

129.7 

July 15, 2001 

159.5 

118.1 


Tables 

Line spread function FWHM for IKONOS panchromatic images (Jan/Feb 2002) 



Easting (m) 

Northing (m) 

MTFC-off 

1.45 

1.27 

MTFC-on 

0.85 

0.52 
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Fig. 1. Big Spring target (August 5, 2001) with MTFC-off (left) and with MTFC-on (right). The 
Roberts gradient magnitude images are shown directly below. Note the sharper gradient in the 
MTFC-on case. The amplitude of the gradient is greater in the latter case. 
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Fig. 2. Illustration of tarp orientation with respect to true north and corresponding example image. 



Fig. 3. Edge spread function projection from angled ground sample interval points. 
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Averaged Spline with data points 



Fig. 6. Pulse response function obtained from blue tarps on July 25, 2001. 


Trimmed input and output 



Fig. 7. Input pulse function and output pulse response. 
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Fig. 12. IKONOS panchromatic images of the edge target tarps deployed at Stennis Space Center, 
Mississippi, for the easting direction measurement on January 15, 2002, (left) and for the northing 
direction measurement on February 17, 2002 (right). Gray rectangular frames overlaid on the tarp 
images show the areas selected for the edge response analyses. 


45 






"'ey 








i rA 


Fig. 13. Measured edge responses (left column) and the best fits to them with superposition of 
three sigmoidal functions (right column). Data are for the measurements in the northing direction 
using the IKON OS image acquired on February 17, 2002, and processed either with MTFC-on 
(bottom row) or with MTFC-off (top row). Respective image areas are shown on the left side of 


the figure. 








Fig. 14. Superimposed edge responses and the fitted sigmoidal functions for the IKONOS images 
processed either with MTFC-on (bottom row) or MTFC-off (top row). Northing direction is the 
left column and easting direction is the right column. 
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Fig, 15, Line spread functions derived from the fitted edge responses for the IKONOS images 
processed either with MTFC-on (bottom row) or with MTFC-off (top row). Northing direction is 
the left column and easting direction is the right column. 
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